Some conservative stopping rules for the operational testing of safety-critical software

Operational testing, which aims to generate sequences of test cases with the same statistical properties as those that would be experienced in real operational use, can be used to obtain quantitative measures of the reliability of software. In the case of safety critical software it is common to demand that all known faults are removed. This means that if there is a failure during the operational testing, the offending fault must be identified and removed. Thus an operational test for safety critical software takes the form of a specified number of test cases (or a specified period of working) that must be executed failure-free. This paper addresses the problem of specifying the numbers of test cases (or time periods) required for a test, when the previous test has terminated as a result of a failure. It has been proposed that, after the obligatory fix of the offending fault, the software should be treated as if it were completely novel, and be required to pass exactly the same test as originally specified. The reasoning here claims to be conservative, inasmuch as no credit is given for any previous failure-free operation prior to the failure that terminated the test. We show that, in fact, this is not a conservative approach in all cases, and propose instead some new Bayesian stopping rules. We show that the degree of conservatism in stopping rules depends upon the precise way in which the reliability requirement is expressed. We define a particular form of conservatism that seems desirable on intuitive grounds, and show that the stopping rules that exhibit this conservatism are also precisely the ones that seem preferable on other grounds

The use of multilegged arguments to increase confidence in safety claims for software-based systems: A study based on a BBN analysis of an idealized example

The work described here concerns the use of so-called multi-legged arguments to support dependability claims about software-based systems. The informal justification for the use of multi-legged arguments is similar to that used to support the use of multi-version software in pursuit of high reliability or safety. Just as a diverse, 1-out-of-2 system might be expected to be more reliable than each of its two component versions, so a two-legged argument might be expected to give greater confidence in the correctness of a dependability claim (e.g. a safety claim) than would either of the argument legs alone. Our intention here is to treat these argument structures formally, in particular by presenting a formal probabilistic treatment of ‘confidence’, which will be used as a measure of efficacy. This will enable claims for the efficacy of the multi-legged approach to be made quantitatively, answering questions such as ‘How much extra confidence about a system’s safety will I have if I add a verification argument leg to an argument leg based upon statistical testing?’ For this initial study, we concentrate on a simplified and idealized example of a safety system in which interest centres upon a claim about the probability of failure on demand. Our approach is to build a BBN (“Bayesian Belief Network”) model of a two-legged argument, and manipulate this analytically via parameters that define its node probability tables. The aim here is to obtain greater insight than is afforded by the more usual BBN treatment, which involves merely numerical manipulation. We show that the addition of a diverse second argument leg can, indeed, increase confidence in a dependability claim: in a reasonably plausible example the doubt in the claim is reduced to one third of the doubt present in the original single leg. However, we also show that there can be some unexpected and counter-intuitive subtleties here; for example an entirely supportive second leg can sometimes undermine an original argument, resulting overall in less confidence than came from this original argument. Our results are neutral on the issue of whether such difficulties will arise in real life - i.e. when real experts judge real systems

Some conservative stopping rules for the operational testing of saftey-critical software

Operational testing, which aims to generate sequences of test cases with the same statistical properties as those that would be experienced in real operational use, can be used to obtain quantitative measures of the reliability of software. In the case of safety critical software it is common to demand that all known faults are removed. This means that if there is a failure during the operational testing, the offending fault must be identified and removed. Thus an operational test for safety critical software takes the form of a specified number of test cases (or a specified period of working) that must be executed failure-free. This paper addresses the problem of specifiying the number of test cases (or time periods) required for a test, when the previous test has terminated as a result of a failue. It has been proposed that, after the obligatory fix of the offending fault, the software should be treated as if it were completely novel, and be required to pass exactly the same test as originally specified. The reasoning here claims to be conservative, inasmuch as no credit is given for any previous failure-free operation prior to the failure that terminated the test. We show that, in fact, this is not a conservative approach in all cases, and propose instead some new Bayesian stopping rules. We show that the degree of conservatism in stopping rules depends upon the precise way in which the reliability requirements is expressed. We show that some rules are 'completely' conservative and argue that these are also precisely the ones that should be preferred on intuitive grounds

Confidence: Its role in dependability cases for risk assessment

Society is increasingly requiring quantitative assessment of risk and associated dependability cases. Informally, a dependability case comprises some reasoning, based on assumptions and evidence, that supports a dependability claim at a particular level of confidence. In this paper we argue that a quantitative assessment of claim confidence is necessary for proper assessment of risk. We discuss the way in which confidence depends upon uncertainty about the underpinnings of the dependability case (truth of assumptions, correctness of reasoning, strength of evidence), and propose that probability is the appropriate measure of uncertainty. We discuss some of the obstacles to quantitative assessment of confidence (issues of composability of subsystem claims; of the multi-dimensional, multi-attribute nature of dependability claims; of the difficult role played by dependence between different kinds of evidence, assumptions, etc). We show that, even in simple cases, the confidence in a claim arising from a dependability case can be surprisingly low

Single stage experimental evaluation of variable geometry guide vanes and stators. Part 1 - Analysis and design

Variable geometry concepts applied to guide vanes and stators in single stage compressor

DIRBE Minus 2MASS: Confirming the CIRB in 40 New Regions at 2.2 and 3.5 Microns

With the release of the 2MASS All-Sky Point Source Catalog, stellar fluxes from 2MASS are used to remove the contribution due to Galactic stars from the intensity measured by DIRBE in 40 new regions in the North and South Galactic polar caps. After subtracting the interplanetary and Galactic foregrounds, a consistent residual intensity of 14.69 +/- 4.49 kJy/sr at 2.2 microns is found. Allowing for a constant calibration factor between the DIRBE 3.5 microns and the 2MASS 2.2 microns fluxes, a similar analysis leaves a residual intensity of 15.62 +/- 3.34 kJy/sr at 3.5 microns. The intercepts of the DIRBE minus 2MASS correlation at 1.25 microns show more scatter and are a smaller fraction of the foreground, leading to a still weak limit on the CIRB of 8.88 +/- 6.26 kJy/sr (1 sigma).Comment: 25 pages LaTeX, 10 figures, 5 tables; Version accepted by the ApJ. Includes minor changes to the text including further discussion of zodiacal light issues and the allowance for variable stars in computing uncertainties in the stellar contribution to the DIRBE intensitie

Evaluating the psychometric properties of the multigroup ethnic identity measure (MEIM) within the United Kingdom

In the present study, we examined the psychometric properties of the Multigroup Ethnic Identity Measure (MEIM) (Phinney, 1992; Phinney & Alipuria, 1990) among an ethnically diverse sample within the United Kingdom. In initial analyses, we evaluated the goodness-of-fit of a one-factor model (i.e., global ethnic identity) and the goodness-of-fit of a two-factor model (i.e., correlated but distinct Exploration and Commitment components). Results of initial confirmatory factor analyses led us to reject both the one-factor and two-factor models. Results of subsequent exploratory and confirmatory factor analyses revealed a three-factor structure (i.e., correlated but distinct Behavioral, Cognitive, and Affective components of ethnic identity) among the sample as a whole (n = 234) and among Asian Indian persons (n = 88) in particular, though resulst were mixed among White U.K./Irish persons (n = 54) in particular. Implications for the study of ethnicity-related concepts in the incerasibgly multi-cultural U.K. are discussed

A Spin-Orbit Alignment for the Hot Jupiter HATS-3b

We have measured the alignment between the orbit of HATS-3b (a recently discovered, slightly inflated Hot Jupiter) and the spin-axis of its host star. Data were obtained using the CYCLOPS2 optical-fiber bundle and its simultaneous calibration system feeding the UCLES spectrograph on the Anglo-Australian Telescope. The sky-projected spin-orbit angle of $\lambda = 3\pm25^{\circ}$ was determined from spectroscopic measurements of Rossiter-McLaughlin effect. This is the first exoplanet discovered through the HATSouth transit survey to have its spin-orbit angle measured. Our results indicate that the orbital plane of HATS-3b is consistent with being aligned to the spin axis of its host star. The low obliquity of the HATS-3 system, which has a relatively hot mid F-type host star, agrees with the general trend observed for Hot Jupiter host stars with effective temperatures $>6250$K to have randomly distributed spin-orbit angles.Comment: 13 pages. Accepted for publication in the Astrophysical Journa

Reliability modeling of a 1-out-of-2 system: Research with diverse Off-the-shelf SQL database servers

Fault tolerance via design diversity is often the only viable way of achieving sufficient dependability levels when using off-the-shelf components. We have reported previously on studies with bug reports of four open-source and commercial off-the-shelf database servers and later release of two of them. The results were very promising for designers of fault-tolerant solutions that wish to employ diverse servers: very few bugs caused failures in more than one server and none caused failure in more than two. In this paper we offer details of two approaches we have studied to construct reliability growth models for a 1-out-of-2 fault-tolerant server which utilize the bug reports. The models presented are of practical significance to system designers wishing to employ diversity with off-the-shelf components since often the bug reports are the only direct dependability evidence available to them